INTERNET
ARCHIVE
Internet Archive:
Universal Access.
Open APIs.
linux.conf.au 2016
Open Knowledge Mini-Conf
VM Brasseur
Ica201 6@vmbrasseur.com
@vmbrasseur
IRC: vmbrasseur
http://vmbrasseur.com
Hewlett Packard
Enterprise
Alexis Rossi
Director of Media & Access
Internet Archive
alexis@archive.org
@alexisrossi
http://archive.org
E— ARCHIVE
Vicky Brasseur
Ica201 6@vmbrasseur.com
@vmbrasseur
IRC: vmbrasseur
http://vmbrasseur.com
Hewlett Packard
Enterprise
Alexis Rossi
Director of Media & Access
Internet Archive
alexis@archive.org
@alexisrossi
http://archive.org
E— ARCHIVE
Rex Feature
Some Handy Links
https://github.com/internetarchive
https://archive.org/details/lca 2016 -ia-apis
https ://www.zotero . org/g rou ps/i nter net_arch i ve_-
_open_apis_and_examples/items
A Tour of Internet
Archive
Internet Archive
501(c)(3) non-profit, founded in 1996
Registered library in California, USA
Partnerships with hundreds of libraries,
museums & universities
24 petabytes of unique data
Where does the money come from?
• 40% digitization projects
• 20% web archiving projects
• 40% foundations & donations
4UM
ERNET ARCH.VE
Photo: Jason Scott
Archived web sites
• 400+ billion web captures, 1996 to present
• Wayback Machine updated within hours
• 700,000 people per day
• 80TB crawl open for bulk download
6
13
20
27
5
12
19
26
1
6
15
22
29
L _
INTERNET ARCHIVE
http://www.gnu.org/
BROWSE HISTORY
http://www.gnu.org /
Saved 4,872 times between January 26, 1998 and June 16, 2014.
PLEASE DONATE TODAY. Your generosity preserves knowledge for future generations. Thank you.
■u
ui.
1997 1996 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013
2014
JAN
FEB
MAR
APR
1
2
3
4
1
1
1
2
3
4
5
7
8
9
10
11
2
3
4
5
6
7
8
2
3
4
5
6
7
8
6
7
8
9
10
11
12
14
15
16
17
18
9
10
11
12
13
14
IS
9
10
11
12
13
14
15
13
14
IS
16
17
18
19
21
22
23
24
25
16
17
18
19
20
21
22
16
17
18
19
20
21
22
20
21
22
23
24
25
26
28
29
30
31
23
24
25
26
27
28
23
24
25
26
27
28
29
27
28
29
30
MAY
JUN
30
31
JUL
AUG
1
2
3
1
2
3
4
5
6
7
1
2
3
4
5
1
2
6
7
8
9
10
8
9
10
11
12
13
14
6
7
8
9
10
11
12
3
4
5
6
7
8
9
13
14
15
16
17
IS
16
17
18
19
20
21
13
14
15
16
17
18
19
10
11
12
13
14
15
16
20
21
22
23
24
22
23
24
25
26
27
28
20
21
22
23
24
25
26
17
18
19
20
21
22
23
27
28
29
30
31
29
30
27
28
29
30
31
24
25
26
27
28
29
30
SEP
OCT
NOV
31
DEC
2
3
4
5
6
1
2
3
4
1
1
2
3
4
5
6
9
10
11
12
13
5
6
7
8
9
10
11
2
3
4
5
6
7
8
7
8
9
10
11
12
13
16
17
18
19
20
12
13
14
15
16
17
18
9
10
11
12
13
14
15
14
15
16
17
18
19
20
23
24
25
26
27
19
20
21
22
23
24
25
16
17
18
19
20
21
22
21
22
23
24
25
26
27
30
26
27
28
29
30
31
23
24
25
26
27
28
29
28
29
30
31
30
INTERNET ARCHIVE
http://www.gnu.org/ BROWSE HISTORY
http://www.gnu.org /
Saved 4,872 times between January 26, 1998 and June 16, 2014.
PLEASE DONATE TODAY. Your generosity preserves knowledge for future generations. Thank you.
. .ki j.uA. ,
1996 1997 1996 1999 2000 2001 2
JAN FEB MAR APR
1234 1 1 12345
56789 10 11 2345676 2345678 6789 10 11 12
12 13 14 15 16 17 18 9 10 11 12 13 14 15 9 10 11 12 13 14 15 13 14 15 16 17 18 19
19 20 21 22 23 24 25 16 17 18 19 20 21 22 16 17 18 19 20 21 22 20 21 22 23 24 25 26
How do we decide what to archive?
• People pay us
• Organizations donate crawled content
• We crawl on our own behalf
• Deep crawl on popular sites
• Broad but shallow crawl on known domains
• Targeted crawls
H
VMBrasseur
INTERNET ARCHIVE
BROWSE HISTORY
432 billion web pages saved over time, donate
.I**'*
B
y—v
I ll\
"&■ ■ 2
U L
0
Tools
Subscription Service
Save Page Now
Wayback Machine Availability API
Archive-lt enables you to capture, marl
Eje and
http://
SAVE PAGE ■
Build your own tools.
search collections of digital content wil
but any
Word Press Broken Link Checker
Banish broken links from your blog.
404 Handler for Webmasters
Help users get where they were going.
technical expertise or hosting facilities.
Archive-lt to build and browse the colled
page as it appears now for
Capture
as a trusted citation in the future.
Only available 'or sites that allow crawlers.
1
30 scanning centers in 8 countries
1 ,000 books per day
;
0
Physical Archive
1 ,500,000 books + plus films, VHS and LPs
Loq in / Sign Uc
SUBJECTS
AUTHORS ADD A BOOK
LISTS RECENTLY HELP
LIBRARY
Search
One web page lor every book.
Show only eBooks More search options
Books to Read The World’s classic literature at your fingertips. Over 1.000.000 free ebook titles available.
Books to Borrow
Here’s a sample of recently returned books from the eBook lending library .
FeIHUC HE
Borrow books at openlibrary.org
2,000,000 items
Feature films, documentaries, commercials,
propaganda, stock footage, cartoons
Archive 65 channels of TV 24 hours per day
TVNews -
OpenLibrary About
HU INTERNET ARCHIVE
Web
Vtcteo Texts Audio Software
Account »
Upload
TV NEWS ARCHIVE Ukraine
C Search inside show
Guide Me
News
In-depth coverage of the
top international stories.
&OQp/n
that's why the show
will be talking about
the indian elections
and economy in the
days and weeks to
come, that is our
show for today. I’m ali
velshi. thanks for
joining us. good
evening, everyone,
this is al jazeera
america. I'm john
siegenthaler in new
&Ofp*n
crisis, and how
swapping blood of the
young with the old
may reverse the aging
crisis, and cinco de
mayo the true story
behind the unusual
holiday. J* we begin in
nigeria where the
anguish has
intensified for the
families of 276 girls
kidnaps three weeks
8. -02pm
about 300 young women
ages 16 to 18 were told
they would be taken
somewhere safe by men
dressed in police uniforms,
but the men were actually
members of boko haram.
we know little about the
girls, only some of their
back our daughters, a
campaign that began
on twitter and quickly
spread, some
protesters in nigeria
say they are being
targeted by the
government, yvonne
ndege says
accusations could stir
up more anger. »
reporter these
allegations could lead
May 5, 2014
8.-00 - 9X)1pm EDT
© Sorrow a DVD
of this stow
GRIP
SCAN
everything possible to
support nigeria and
return the young pern
to their homes. »
how they disappeared
without anybody
seeing anything has
been the question all
along, nigeria has
been battling boko
haram for years now.
the group's name
means western
group. » boko haram
is a frig movement, it
is a terrorist group
which has
assassinated muslim
learned men,
terrorized
communities, and it is
a small group
relatively speaking
but casts large
shadow. » reporter
its latest move
ra
cinfi il
n A
Audio
• 2,500,000 items
• 150,000 live concerts from 6,000 bands
• 60,000 netlabel releases from 2,000 labels
• Mirrors of Jamendo, IUMA
• Audio books, radio shows, podcasts